In virtual reality or augmented reality applications, it is often helpful to be able to render a scene from arbitrary points of view, allowing free-viewpoint navigation for example. While conventional computer graphics allow synthesis of modeled scenes from arbitrary points of view, the photorealism of natural scenes using models is elusive, at least without extreme computation, especially in the presence of complex material and lighting phenomena such as reflection, refraction, and scattering. Light fields aim to provide photo-realistic renderings of three-dimensional (3D) scenes from a range of viewpoints, even in the presence of such complex material and lighting phenomena, to enable rich and immersive viewing experiences.
A light field is a set of rays passing through space (e.g., passing through a point in space), each ray bearing a color, which may be specified by a set of color components (e.g., tristimulus color components, such as triple of red, green, and blue values). If all rays passing through a 3D space are known, it is possible to synthesize an image as if it were taken from a camera at an arbitrary viewpoint. A surface light field is a function representing a light field of rays passing through multiple points on the 3D surface of a 3D object. For each point on the surface, the surface light field around that point, as a function of viewing direction (e.g., expressed in spherical coordinates, such as, azimuth and elevation) can be represented by a corresponding “view map,” which may take the example form of a spherical image or other data structure that correlates a set of directions (e.g., viewing directions) with respectively corresponding associated colors.
Thus, the 3D surface of the 3D object can be represented by a set of points (e.g., a point cloud) that each have a corresponding view map. As used herein, a “view map” for a given point is a set of viewing directions and their associated attribute values (e.g., colors) which collectively specify how the point appears when seen from those viewing directions. A set of view maps corresponding to multiple points across the 3D surface can be used to render the 3D surface from any viewing direction. Accordingly, to communicate a representation of the surface light field of the 3D object, a first machine may be configured to provide a second machine with the set of points and the set of view maps, and the second machine may be configured to receive this information and render 3D object based thereon.